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[57] ABSTRACT 

This disclosure provides a system for editing graphical data 
based upon relative time of entry, and in particular, provides 
editing software used with an electronic digitizing clipboard. 
A-stylus of the clipboard is used to write on pages, with a 
digitizer underneath the pages generating an electronic 
record of entered marks for storage and later recreation of 
the pages. The-electronic record represents both stylus 
spatial coordinates, normally sampled every ten 
milliseconds, and certain user commands such as page 
identification, inserted at significant locations within the 
aggregate data stream. Data in the stream is associated with 
prior page identification commands, such that subsequent 
data may be cut or sorted based upon timing of that data 
relative to the page identification commands. In this manner, 
the remote editor, and splicing software on the editor, may 
distinguish data based upon time of entry, and move that 
data notwithstanding any spatial overlap with other stylus 
data. 

16 Claims, 7 Drawing Sheets 
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SYSTEM FOR EDITING GRAPHICAL DATA Unfortunately, the principal embodiment shown by the 

BASED UPON RELATIVE TIME OF ENTRY '499 Patent requires a user to correctly and timely switch 

definition of the current page so that the electronic iofor- 
The present invention relates to a system for editing mation is associated with the proper page. Failure to prop- 
graphical data based upon relative time of entry. In 5 erly switch page definition could potentially result in infor- 
particular, it provides a device that processes a sequence of mation being associated with the wrong page; if the user did 
data in a manner such that data can be filtered and assigned not properly switch page definition, then data intended for 
to different pages of graphical data based upon relative time different pages might be incorrectly associated with a single 
of entry, notwithstanding spatial overlap of the data. page, in a spatially overlapping manner. The '499 Patent also 

10 does not teach any way of editing entered data. 

BACKGROUND definite need exists for a system for a mechanism for 

Electronic data capture devices have recently gained significantly reducing the impact of errors caused by data 

popularity as an effective way to electronically capture and transposed between multiple forms. A definite need also 

index data. For example, delivery services, retail stores, exists for a system that provides for ready editing of entered 

manufacturers and others commonly use such electronic electronic data. Ideally, such a system should be usable not 

devices to partially or completely replace data entry on o^l/ ^ith handwntmg capture devices, such as those 

conventional paper forms, so that data may be stored referred to above, but should be applicable to a broad range 

electronically, saving space associated with records reten- devices and appUcaUons that create data represenUng 

tion. To cite another example, many retail stores now use graphics or spatial coordinates. The present invention solves 

signature capture devices to process credit card transactions; ^hesc needs and provides further related advantages, 

a customer signs his or her name upon a paperless electronic SUMMARY OF THE INVENTION 
capture device, in lieu of using a carbon form. A hardcopy 

receipt bearing a reproduced signature and purchase record .The present invention solves the aforementioned needs by 

is then printed for the customer, while the retail store retains providing a system that processes a sequence of data in a 

only an electronic record of the transaction. ^ manner such that data can be filtered and assigned to one of 

Most such electronic capture devices are designed to be multiple pages based upon-relitiveniiDetGfireBtry, notwith- 

relatively foolproof, e.g, such that entered data is automati- ^landing any spatial overlap of data. In particular, the present 

cally applied to the correct document, and litUe possibility invention provides machine readable instructions that permit 

exists for transposing signatures or data to the wrong trans- filtering of data in much the same manner as one might edit 

action. In the case of capture devices controlled as one ^^^^^ tape, e.g, by displaymg entry of data with respect to 

element of a multiple step process (e.g., credit card Ume and providing an operator with play forward and cut 

processing), error is typically controlled by processing only capabiHtics. As a result, the present mvention provides an 

one transaction at a time and accepting a customer signature system that can be mexpensively implemented in 

only at a specific time in the process so that the signature is 35 "^^r^y graphics processing environment, e.g., on per- 

unambiguously associated with the proper transactional sonal computers. 

data. Similarly, in the case of portable devices used in A first form of the invention provides instructions for 

inventory tracking and similar applications, capture devices controlling a machine, stored on machine readable media, 

arc frequently designed to be used with only one form at a These instructions (e.g., software, firmware, or another 

time. 40 method of controlling a machine) permit an operator to edit 

While generaUy useful for their intended purposes, these ^^^^ ^P°° relative times associated with that data. The 
electronic devices (and many others intended to electroni- instructions associate the data with a first graphics display 
cally replace conventional forms-processing practices) often window, and accept inputs from the operator defining rela- 
suffer from being somewhat rigid, for example, in not tive time parameters to define a subset of data. The instruc- 
permitting editing of entered data. Further, devices required 45 separate the subset of data from the data, to leave 
to internaUy store electronic data (e.g., portable devices) remaining data, which can be displayed in the first graphics 
must frequenUy contain substantial internal memory, mak- display wmdow. Innhis manner, the subset of data (e.g.. 
ing such devices costly and heavy. In many conventional individual stylus strokes, or groups of or portions of them) 
paper processing applications, it is generally desired to ™^y removed from the remaming data based upon their 
aUow a user to move around and concurrenUy handle 50 associated relative times, notwithstanding any spatial over- 
multiple forms, as one might do while completing standard ^^P i° spatial coordinates of the data, 
hardcopy forms. In more particular aspects of this form of the invention, 

RecenUy, U.S. Pat. No. 5,629,499 (the "M99 Patent") has ^« data can be stylus stroke data, representing continuous 

provided a partial solution to the problem of concurrent stylus strokes and sampled at regular and periodic intervals 

processing of multiple paper pages or forms, via that 55 (^ 8- milliseconds when the stylus is being used), 

patent's disclosure of a mulfiple page, portable, digitizing '^^ instmctions may operate in a manner that causes the 

clipboard. Using the clipboard, a user can actually write machine to display data according to time of entry, such that 

upon multiple, stacked pieces of paper, with a digitizer operator can play data forward and backward, and 

beneath the paper capturing pen stroke information; a user sunulate entry of data upon a display monitor at approxi- 

interface is utilized to switch a displayed indication of the 60 "lately the rate at which data was originally entered, 

current page between the multiple pages, and captured A second form of the invention provides a method which 

electronic data is thereby associated with the selected page. performs similar tasks to the instructions just described. 

Thus, a user can simultaneously enter data on several The invention may be better understood by referring to 

hardcopy pages, all stacked upon the clipboard, while the the following detailed description, which should be read in 

cupboard simultaneously and automatically also creates 65 conjunction with the accompanying drawings. The detailed 

electronic information corresponding to pen strokes for description of particular preferred embodunents, set out 

archiving or recreating each hardcopy page. below to enable one to build and use particular implemen- 
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tations of the invention, is not intended to limit the enumer- 
ated claims, but to serve as particular examples thereof. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 shows a digital electronic clipboard including both 
a digitizer section and a stylus, which has an inking tip at one 
end; the clipboard is seen as mounting two stacked pages 
(one of which, the top page, has a comer upturned). 

FIG. 2 is a diagram showing a logical hierarchy of forms 
organization of the clipboard of FIG. 1. 

FIG. 3 is an illustrative diagram indicating data process- 
ing elements of the clipboard of FIG. 1, and how those 
elements cooperate to create a data stream for selective 
transmission to a remote editor (e.g., a remote personal 
computer). 

FIGS. 4-8 illustrate the preferred data compression 
scheme used by the clipboard of FIG. 1, with FIGS. 4-^ each 
representing one data packet consisting of either one com- 
mand code or a set of spatial coordinates of the stylus^ tip; 
as seen in FIGS. 4-8, the data compression scheme used by 
the clipboard results in each data packet occupying two to 
six bytes. 

FIG. 4 is a diagram used to illustrate the format of one 
packet of uncompressed data {x,y,p}, with two bytes of "x" 
data, two bytes of "y" data, and two bytes of "type*' data 
represented by the packet; the three first bits of the first byte 
indicate that fiill value "x" data, data and "type" data are 
present in the packet. 

FIG. 5 shows a data packet similar to that of FIG. 4, but 
without any "type" data, as indicated by a logical "zero" in 
the high-order bit of the first byte. When the data packet 
represents stylus data, as opposed to a code, non-inclusion of 
the "type" data field indicates that there has been no change 
in data type. 

FIG. 6 shows a data packet similar to that seen in FIG. 5, 
but in which "x" data is compressed to one byte, such that 
the packet includes one byte of "x" data, and two bytes of 
"y" data. Compression of "x" data to one byte is indicated 
by a logical "one" in the second highest order bit of the first 
byte. The compressed "x" data represents a six bit signed 
oflfeet from the "x" value of the previous data point. 

FIG. 7 shows a data packet similar to that seen in FIGS. 
4-6, but in which "y" data is compressed to one byte, such 
that the packet includes two bytes of "x** data, followed by 
one byte of "y" data. Compression of "y" data to one byte 
is indicated by a logical "one" in the third highest order bit 
of the first byte. The compressed "y" data represents a seven 
bit signed offset from the "y" value of the previous data 
point 

FIG. 8 shows a data packet similar to that seen in FIGS. 
4-7, but in which each of the "x" data and the "y" data are 
compressed to one byte (indicated by logical "ones" in the 
second and third bits of the first byte, respectively), and in 
which no "type" data is present (indicated by a logical 
"zero" in the first bit of the first byte). 

FIGS. 9-lS are used to indicate processing of data by a 
remote computer to separate spatially overlapping data (in 
electronic form) into two separate image pages based upon 
relative time of data entry. FIGS. 9-10 each show writings 
entered using the clipboard of FIG. 1 where it is presumed 
that a user forgot to electronically indicate a page change 
using the user interface of the clipboard; in FIGS. 9-10, the 
user has first entered a dollar amount upon a top page (FIG. 
9), then switched to a bottom page (FIG. 10) to enter the text 
"Have a nice day," and lastly, switched back to the top page 
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(FIG. 9) to enter two initials. FIGS. 12-18 represent a 
"splicing" program used by a remote computer, to separate 
the spatially overlapping data and form two separate pages 
fi-om that data to correspond with what was originally 
5 entered on the pages on the clipboard. 

FIG. 9 represents aggregate writings upon a top page 
mounted by the clipboard, respectively, a dollar amount and 
two initials, 

FIG. 10 represents aggregate writings upon a bottom page 
mounted by the clipboard, namely, the text "Have a nice 
day" (entered after the writing of the dollar amount on the 
top page indicated by FIG. 9, but before entry of the two 
initials upon the top page indicated by FIG. 9). 

FIG. 11 represents an image formed from electronic data 
representing both pages of FIGS. 9 and 10, where there has 
been no intervening command to identify the transitions 
from the top page to the bottom page, and back again; the 
captured electronic data consequently indicates spatially- 
^ overlapping data aU on a single page. 

FIG. 12 shows a screen of a "splicing" program run upon 
a remote personal computer; a status bar at the top of the 
FIG. 12 indicates that time is "early" in the writing process 
represented by FIGS. 9 and 10, e.g., corresponding to a time 
25 prior to entry of any writings on either the top or bottom 
pages. FIG. 12 also indicates the presence of "fast rewind," 
"play backward," "play forward," "fast forward," "pause," 
"mark*' and "cut" buttons, which are displayed at the bottom 
of the screen. 

30 FIG. 13 is a representation similar to FIG. 12, but in which 
the "play" button has been pressed and some time has 
passed, as indicated by the status bar at the top of FIG. 13; 
the sequence of data entered using the clipboard has been 
"played forward" to correspond to the time just after entry 

35 of the written amount ($69.95), upon a graphical represen- 
tation of the top form (from FIG. 9). 

FIG. 14 is a representation similar to FIG. 13, but in which 
more time has passed, corresponding to the writing of both 
of the amount ($69.95) upon the top form (FIG. 9) and the 

^0 text "Have a nice day" upon the bottom form (FIG. 10), 
without the user having input a page identification command 
through the clipboard's user interface. Hie operator has at 
this point in time selected the "pause" button, just after entry 
of the text "Have a nice day;" in addition, the operator has 

45 also selected the "mark" button, to mark a reference point in 
time. 

FIG. 15 is a representation similar to FIG. 14, but in which 
the "play backward" button of the computer screen is 
illustrated as selected, to scroll writings with respect to time 
backward prior to commencement of writing "Have a nice 
day." 

FIG. 16 is a representation similar to FIG. 15, but in which 
the "play backward" button has been deselected and the 
J J "pause" button has been highlighted, so freeze writings at a 
point just after entry of the amount ($69.95). 

FIG. 17 is a representation similar to FIG. 16 in which the 
operator has now selected the "cut" button, thereby remov- 
ing all writings in reverse time sequence from the "mark" 
go point to the current point, and entering them into a "cut" 
window. 

FIG. 18 is a representation similar to FIG. 17, but in which 
the operator has now placed cut writings into a new graphics 
window, identified as "page 2", and has also selected the 
65 "play forward" button of the first graphics display window, 
such that the subsequent initials ("tj," entered later in time) 
will be placed into the first window together with the 
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monetary amount ($69.95), to correspond to the hardcopy, user is writing on. The system may also include a bar code 

top form (FIG. 9). scanner (not shown in FIG. 1) for scanning a bar code 27 

mounted upon each page each time the user newly writes on 

DETAILED DESCRIPTION the page having the bar code. The bar code, in this 

Ibe invention summarized above and defined by the 5 jmplemcntatioQ, would contain information about form 

enumeratedclaimsmay be better understood by referring to J^Pf' 7°^,^"^. ^ weU as a unique serial code (form 

- „ . J , -1 J J • u- t- u UL J . instance 1 which distinguishes a completed hardcopy 

the following detaded description, which shouM be read in ^^^^ "invoices." It should be recognized that 

conjunction with the accompanymg drawings. Tins detailed cUpboard U need not be used only with pages having 

descnption of a parUcular preferred embodiment, set out preprinted information (e.g., "invoice") and that blank pages 

below to enable one to build and use one particular imple- may be used as well and identified by a bar code or identifier 

mentauon of the mvention, is not mtended to bmit the in precisely the same maimer. Details of a bar code scanner 

enumerated claims, but to serve as a particular example g^^^ble for use with the chpboard 11 may be obtained in 

Oiereof. The particular example set out below is the pre- ^^f^^^^^ copending application Ser. No. 08/962,066, for 

fcrred specific implementation of a data entry system, , "Handheld Writing Device and Related Data Entry 

namely, a digital electromc chpboard and a remote editor ^5 ^ „ ^^^^ ^^^^ ^^^^ disclosure, by 

and sphcmg software. The mvention, however, may also be inventors Robert Leichner, Bo Curry and Dan Flicldnger, 

applied to other types of systems as weU. ^^-^^ ^^^^^ incorporated by reference in its entirety, as 

I. INTRODUCTION TO THE PRINCIPAL PARTS ^^^^ 

20 context of the present invention, page change 

The present invention is embodied in a digital electronic commands and data from the stylus and digitizer are inte- 

cUpboard 11 having special appUcations to page and form grated into a single, sequenced data stream, which represents 

processing systems. U.S. Pat. No. 5,629.499 (the "M99 both user commands and stylus data. The stylus 15 is 

patent") sets forth background relating to a clipboard design sampled by cHpboard electronics each ten milliseconds, to 

utilized in the preferred embodiment, and is hereby incor- ^ produce a continuous stream of stylus position data, repre- 

porated in reference in its entirety, as though reproduced scnting "x" and "y" spatial coordinates with respect to the 

cl^board, as well as pressure upon a writing tip of the stylus. 

With reference to FIG. 1, a digital electronic clipboard 11 This pressure indication in the preferred embodiment 

is illustrated as including a digitizer section 13, which includes a one-bit tip-down signal, indicating whether the 

supports a top page 14 (e.g., blank paper or a preprinted 3Q writing tip is depressed against a page (e.g., the stylus is 

document having data entry fields), a stylus 15 for writing on being used to "write"), but could also be made to occupy 

pages and generating electronic stylus data, and a tether 17, more bits to represent relative pressure, or "darkness" of 

which physically and electronically connects the stylus to each pen stroke upon the page. 

the clipboard. In the context of the present invention, it is While a user of the clipboard 11 may be presented with 

desired to provide a user with the ability to concunently 35 several options for selecting page change commands (e.g., 

process multiple forms, and to this effect, a corner 18 of the via the bar code scanning option, discussed above), the user 

top page is seen as "lifted" to reveal the presence of a interface 25 preferably may always be used to select com- 

second, bottom page 20, beneath the top page. The clipboard mands. This selection is performed by using scroll buttons, 

may be utilized for writing upon each page 14 or 20, and to scroll through descriptions for different forms or pages, 

switching between them as desired, to both complete entry 4^ which are individually displayed for the user. An "enter" 

of data on paper forms and simultaneously generate elec- button is then utilized, while a selected description is 

tronic records of all information written upon each page. displayed, to trigger insertion of an identifier for the desired 

The clipboard 11 further includes a clip mechanism 21, page or form into the data stream, as has just been described, 
for pinning the pages 14 and 20 to a paper supporting surface In connection with the "type" data, user commands from 
23, and for permitting ready addition of a large number of 45 the user interface as well as certain other commands are 
additional pages, all in stacked relationship; FIG. 1 also inserted sequentially into the stream of stylus data as escape 
indicates use of a built-in microphone 24 which may be characters; that is to say, the commands are inserted as soon 
optionally used to input audio data Hnked to specific forms as possible into the data stream, e.g., immediately, if no 
or specific fields appearing on preprinted forms. The stylus stylus data is being concurrently processed, or immediately 
has a writing tip 19 (such that the user marks the pages, e.g., 50 after a sample of stylus data which is concurrently being 
by ink, pencil or otherwise), a pressure sensing mechanism, processed. A resulting aggregate data stream, including 
and position sensing electronics which generate signals certain commands which have been merged with stylus data, 
representing location of the writing tip 19 with respect to the is sequentially stored in memory internal to the clipboard 11, 
digitizer section 13. The stylus 15 may further include a Ught for later upload to a remote editor (preferably, a remote 
emitting diode ("LED") 35, which is IQuminated when the 55 computer nmning suitable data editing and clipboard inter- 
writing tip 19 is pressed against a page (e.g., when the stylus face software). If the inserted command is an identification 
is being used to write upon the page). of a particular form (e.g., the bottom form 20), the position 

While the user may enter data on hardcopy pages in any of tbat command within the aggregate data stream is tem- 

order desired, the clipboard 11 is preferably "informed" porally significant and all electronic data following that 

when the user ceases writing on one page 14 or 20 and 60 identification in the memory can be associated with the 

begins writing upon the other page 20 or 14. To this effect, bottom form, until another electronic identifier indicates a 

a number of mechanisms may be implemented to electroni- change to another page. 

cally mark a page change in the electronic records. For The electronic identifier (for the page change) which is 

example, the preferred clipboard 11 includes a user interface inserted in the data stream should preferably specifically 

25, which has an electronic display and buttons for display- 65 identify the type of form utilized, such that any preprinted 

ing and selecting specific user commands from a predefined mformation can be electronically merged with entered stylus 

menu, such as for notifying the clipboard which page the data once the information has been transferred to the remote 
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computer. For example, if a user of the clipboard U is include most anything input using an "enter" button of the 

completing an "invoice", the electronic data generated by user interface, such as selection of a new page for writing, 

the clipboard preferably includes a code that enables a creation of a new form "instance," deletion of a page from 

program running on the remote computer (not seen in FIG. the clipboard's active memory, or upload of data from the 

1) to electronically associate an image of a blank invoice S clipboard's serial memory to the remote computer. A com- 

with the electronic data; that is to say, the clipboard gcncr- mand from the user might also take the form of a scanned bar 

ates data which can later be combined with an electronic code of the form, or of a package, if a bar code scanner is 

imageof a form. In addition, use of a code representing both connected to the clipboard. Finally, commands from the 

form "type" and "instance" permits the remote computer to remote computer might include a request for retransmission 

automatically merge entered data with previously entered lO of data, download of additional forms definitions to the 

data for the same page or form; a user can start filUng out a clipboard, or a download of software to be run on the 

form during one clipboard session, upload data to the remote clipboard's CPU. Communication between the remote com- 

computer, and then complete the same page or form during puter and the clipboard can occur via any mechanism, such 

a second session, and the software of the remote computer as infrared communications but, preferably, the clipboard 

can associate the data with the same page or form. Specific 15 primarily relies upon a serial communicatioos port (such as 

codes for each page or form are not the only mechanism for a RS-232 or similar standard computer interface) for com- 

tracking forms, and the clipboard can also use an index munication with the remote computer, 

which merely identifies active pages upon the clipboard, [q the context of the present invention, commands iden- 

e.g., pages 1-132. In the preferred embodiment, both the tifying that the user desires to write on or create a specific 

index and specific codes are used by the clipboard and 20 form instance (and page, if appropriate) are directly inserted 

remote computer to track and process pages and forms. As into the stylus' data stream, at a significant location, 

used herein, the term "form" includes both preprinted and Preferably, a number of pre-existing forms arc known to the 

blank forms which may have one or more "pages." clipboard, and new forms can be created, all as defined by 

The preferred clipboard embodiment generates and orga- a forms hierarchy (such as described below in reference to 

nizes entered data in such a manner that it can easily be ^ FIG. 2). Switching between known forms can be effected 

edited after generation, even if a user forgets to mark, or using the CPU's active forms list, by using the user interface 

improperly marks, a page change. la this context, it is 25 to select one of multiple pages on the clipboard and by 

desired in many applications to have a flexible system usable selecting that page by pressing an "enter" key when the 

in creating and marking many different pages. Hie preferred appropriate selection is displayed. If a bar code scanner is 

clipboard meets this need by generating a single data stream, 30 ^s^d to scan a bar code representing a page, the CPU is 

representing all data entered using the stylus 15, and certain instructed by firmware to first search the active forms list, to 

commands entered by the user, sequentially arranged within determine whether a scanned bar code corresponds to an 

the stream for easy later sorting or editing of stylus data. Via active form and, if not, the CPU then automatically creates 

software on the remote computer, stylus data can then be a new form (adding pages and form type definition to the 

displayed in an image window dedicated to each form. The 35 active forms list). If the user is manually utilizing the user 

software permits storage of the displayed data only, or interface 25 to signify addition of a new page to the 

merging of the displayed data with image information rep- clipboard, the user selects a "create form" menu option, and 

resenting the "type" of form (e.g., an image of a blank form type, instance and page menu options, as exemplified 

"invoice") for recreation of the form. Importantly, as farther below in connection with a discussion of FIG. 2. Commands 

described below, the remote computer preferably also runs from the user interface 25 selecting or creating pages are 

"splicing" software which permits processing of stylus data executed (and inserted into the stylus* data stream) at the 

based upon relative time of entry, such that stylus data can time that the user presses the "enter" key. 

be distinguished, cut and reassigned between multiple pages. Specific commands selected by the user are not the only 

notwithstanding any spatial overlap with other (correcfiy lypg of commands which are inserted into the data stream; 

assigned) stylus data. as an example, each time the stylus is newly depressed 

Prior to further explanation of the functional configuration against a page, a time code is automatically inserted into the 

of the preferred clipboard, it will be helpful to explain the data stream, indicating both date and time of day. 
nature of some commands used in the preferred 

embodiment, as well as preferred forms hierarchy for varied B. DEFINITIONS OF FORM TYPE AND 

applications of the clipboard. Use of a forms hierarchy, such INSTANCE 

as that described below, permits ready adaptation of the ^ , . . ^ . , . , . 

clipboard to certain commercial uses where defined, pre- \ ^ illustrates a hierarchy 37 of hypothetical forms 

printed forms are commonly used. chpboard s memory, a hierarchy which can be 

tailored to a specific application. For example, FIG. 2 
A, COMMANDS illustrates commonly used business forms, including, "pur- 
Commands can be generated automatically by the chase order" 39, "invoice" 41, "release" 43 and "new form" 
clipboard, by the user (through the user interface) or by the 45 (i.e., blank or unknown pages). The listed hierarchy 
remote computer. One example of a command automatically would be used to either define a new form or activate an old 
generated and inserted directly into the data stream by the form and cause the CPU to add the form to its list of active 
clipboard is a time stamp, which can be inserted into the data 60 forms. 

stream with each new stylus "stroke" (e.g., each time In using the user interface 25 to access the stored hierar- 

"pressure" on the writing tip indicates new contact between chy of predefined form types, a user would scroll through 

the stylus and a page); a time stamp could also be generated root menu commands such that the word "forms" 47 is 

by- a clipboard central processing unit ("CPU") with each displayed by a Liquid Qrystal Display ("LCD") of the user 

■sampling of stylus data, and formed as part of the original 65 interface, and would push an "enter" button. The user would 

stream of stylus data sampled and fonnatted by the CPU. then scroll through predefined forms "purchase order" 39, 

Examples of a command generated by the user would "invoice" 41, "release" 43 and "new fonoa" 45, and would 
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push the "enter*' button again to select whichever selection used to upload, display, process and edit entered stylus data, 

is displayed upon the LCD. Having thus selected general The remote computer can also be used to prepare the data for 

form "type," the user preferably then begins another menu archiving, as indicated by the presence of a computer disk 
selection process, where the user can select either an existing 

"instance" 49 (e.g., a previously commenced form already S FIG. 3 shows the stylus 15 and paper supporting surface 

having data) or a new instance 51 (uncommenccd, blank 23 (which includes the digitizer), the user interface 25, 

form). Upon selecting a new instance, a new instance several boxes indicating CPU and support chip processing 

number is assigned to the new form (e.g., existing release ftmctions, and memory used by the clipboard, generaUy 

number two), etcetera. If the form is a type which includes designated by the reference numeral 65 and shown m two 
multiple pages, the user is presented with another menu that 10 ^''''^^ ^""^ ^j*- Memory used for data storage is preferably 

permits the user to select a particular one of the multiple non-volatile "flash" memory and, preferably, the clipboard 

pages, and so on, as indicated by the reference numeral 53 includes both internal, permanent memory (ideally at least 

in FIG, 2. As further indicated by reference numeral 55 of megabyte) with an external slot for receiving a PCMCIA 

FIG. 2, a user can define a new form (e.g., a blank page or ^ard having additional memory. The first memory box 67 
previouslyundefined form type) consisting ofeither a single 15 represents memory space reserved m random access 

page or multiple pages, as designated by reference numerals memory ("RAM") for CPU functions, while second memory 

57 and 59 respectively fizsh. memory used for sequential storage of data 

^ ^ xi.-iij-ij and selected commands ("data memory"). FIG, 3 also indi- 

Data entered onto forms are not electronically displayed , ^ ^ ^ -- -^^n ul- cui-ij 

... .ij.r -1 ^ cates a data transmission port 71, which preferably includes 

to the user, nor is stylus data from a previously commenced . , . ^. r - * -^l 
- , ' ., , , . 5 1- i_ t, ■ . 1 in a senal communications port for communications with the 

form instance necessarily stored in the clipboard s mternal ^ * /• j- * j u r i -ti\ 

„ , :. , ^ , ^ . remote computer (indicated by reference numeral 73). 

memory. Rather, each tune a page is newly created or ^ r t - - ^. , 

activated, an index is created in a memory space reserved by J^^ P^^^^"^^ i^er mterfacc is indicated in HG, 3 by the 

the clipboard's CPU for use in processing various com- ^^^^^^^^^ ^5. A LCD 75 is used o display indi- 

mands. Completed form instances preferably are deleted ^' ^^.^^ "^f 

f,«™ r.^r^^ ^ fr.^ ^^r.^ o r,^ 25 dowu buttons 77 are utilizcd to scroll through dispUyable 

rrom memory once a lorm or page is completed and ^ 7, „. 

1 A A*^ ♦k^. *^ . /Jr^ n ^^^r.^A menu choices. The interface also mcludes an enter^ button 

uploaded to the remote computer (e.g., either via a command -r>i- t j - i , . • ^ 

\ . - ,1 * c „ *u „ * ™ t - .* 79 for selecting displayed menu choices, as well as a menu 

automatically sent from the remote computer, or via selec- , ^ ^ . ^ , , ,„ . 

* *u» button for retummg to the root menu, a send button for 
tive use of the user mterface). . . , , ,u ♦ « ^ «j i » » 

. , ^ transmittmg data to the remote computer, and a delete 

For example, when a user imtially begms a new form button for removing form instances from the memory space 

instance, data entered for the new form mstance will be reserved for the CPU (the three latter buttons are coUectively 

stored in the clipboard's mlern^ memory (together with designated by the reference numeral 81 in FIG. 3). 

command codes). A page identification code will also be i^^portantly, transmission of the single data stream of com- 

gerierated and stored m the CPU s reserved memory space ^^^^^ commands to the remote computer 61 is 

and m the data stream Subsequendy, the user can opUonaUy transmission that can occur, and the 

upload the contents of the sequential memory corresponding ^^^^^^ ^ preferably used (via firmware for the 

to the page or form to a remote computer, and erase the ^Hpboard) to place the clipboard in a state where it can 

correspondmg data, such that the clipboard no longer has ^^^^-^^ ^^^^ ^^^^^^ commands and programming relating 

memory of entered data; the chpboard does, however, selec- definition of form types. 

tively retain memory of the form identification code of the a*.l * ^ • * *l ^t^ti 
-i^ c • . L • t- I. i_ ^ J A 40 At the time a user creates a new form mstance, the CPU 

specific form instance which has been created. Asa . -j* l rr l-l*!^ 

^ L *i 11 r assigns an index to each page of a form, which it stores m 

consequence, the user can subsequently recall the form ° , 7^ • i- ^ c ^ r j 

• . . jj jj-*- 1 J * u • .1- * J • its reserved memory space 67 in a list of active forms, and 
instance to add additional data, which is then stored m the „„t r « . ^ . - . . / r - 

, J, 1-1. 1- I. J J * J- I . J the CPU also assigns a unique code to each page (if one is 

clipboard s memory. The clipboard does not display entered . i j i * *u i - l j K ^ 

, f . . . 1 -J . « •* * *t. . . not already known to the clipboard, e.g., via memory or 

data, but simply identifies it to the remote computer as ji_ j\t^ i i .jc 

^ J. / -n c • . TTnL 45 scanned bar code). For example, a newly created form might 

correspondmg to the speanc form instance. When the . . , ^ cut « n j .-c j 

^ r.L T * 1 J J * *i_ receive an index of 1, as well as an identification code, 

contents of the sequential memory are agam uploaded to the • i ^ -j r w • »\ j • 

... J J • ^ J • * mcludiog a type identifier (e.g., invoice ) and a unique 

remote computer, the page identification codes mscrted into . , ° . i.' l *• r 

1 J L * * * serial code. The user, m switching between active forms, 

the sequential memory are used by the remote computer to , , ,^ • . r u* i_ j- J 

• - 1 1 J J J f -.1 1 J J does not need to navigate the forms hierarchy, discussed 

combine the origmal uploaded data with the new uploaded . . . , „ ... ^/jl . . r 
, . , .1 L. * 1 .1 111- * 1 * 50 earlier, but can simply scroll through the CPU s mdex of 

data. .Importantly, each stylus stroke will have at least one % , A. « * _jj i -rn * •* i_ u *„ 

..S.. "^i- , J j-rr . . 1 .1 active fonms and use the enter key 79 to switch between 

associated time of entry, and so, different stylus strokes are ... , * i j i- u j> -r * 

J 4 Au ,u multiple pages stacked on the clipboard's surface. Data 

distinguished by time of entry and can be separated by the * j • *u .i ic u L '*j *i. 

„ Z ^r.A .L^.i^^A i« ^;ffo««t entered using the stylus 15 will be associated with the 

Temote computer and be reassigned to different forms, , - j*.^ 

^ ^ . J- .-11 currently selected page, and the stylus position IS detected 

notwithstanding any spatial overlap. i . r j i • l ^ 

^ ^ ^ 55 cvenif there arc a large number of underlying pages between 

11. PREFERRED HARDWARE FUNCTIONAL P^P^^ supporting surface and the form being written 

CONFIGURATION ^P°° ^s^^* 

Each time the writing tip 19 is newly depressed, the 

Physical and electronic configuration of the preferred clipboard's CPU samples a real time clock 83, and inserts 

device will now be additionally explained with reference to this data (including date and time of day) into a data stream 

FIG. 3. firom the clipboard and stylus. The clipboard is normally 

The clipboard and its internal, functional components are maintained in a sleep mode, and is awakened when the user 
indicated by the reference numeral 11 of FIG. 3. FIG. 3 also places the writing tip in contact with an object, thus closing 
indicates use of remote transmission, e.g., modem, infirared a switch in the stylus which triggers an interrupt signal 
communication or other serial communications, to a remote 65 within the CPU. This interrupt signal causes the CPU to 
computer 61. The remote computer, preferably a personal empower clipboard electronics and begin actively "listen- 
computer rurming suitable imaging and interface software is ing" to the stylus. When "listening," the chpboard preferably 
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samples data each ten milliseconds, although only data found in U.S. Pat. No. 4,806,918, which is hereby incorpo- 

representing writings on the clipboard (e.g., when the tip- rated by reference in its entirety as though identically set 

down switch is concurrently closed) will be passed to the forth herein. 

data memory 69 as a stream of stylus data. At the same time b. DATA COMPRESSION AND STORAGE IN 
that the clipboard processes stylus data, it compresses that s MEMORY. 

data as appropriate, as indicated by block 87 in FIG. 3. ^ mcnUoned earlier, it is desired that the preferred 
Each time a user selects a page (or creates a new page or clipboard be lightweight while having memory sufficient to 
form) using the user interface, the CPU uses the index of store data corresponding to entry of a large number of forms, 
active page and forms to retrieve from reserved memory xbe clipboard's CPU thus, when processing commands and 
space the unique identification code corresponding to a ^^^^ ^ single data stream, also compresses the data 
particular page, and the CPU inserts this identification code for sequential storage in the memory. The memory, generally 
(as indicated by reference numeral 89) into the data stream indicated by reference numeral 65 in FIG. 3, preferably 
via a summing junction, as has been previously described. includes both at least one megabyte of nonvolatile memory 
The aggregate data stream is then sequentially stored in data internal to the clipboard, and it may also include expandable 
memory 69, with memory location indexed by form ^5 jj^gj^Q^ sy^h as a sixteen megabyte PCMCIA flash card, 
instance, page and time of entry also being recorded in the jhe clipboard's CPU reserves a space in memory 67 for 
CPU's reserved memory. indexing currently active forms and for also tracking 
In processing other user commands, the CPU relies upon memory contents in 32 kilobyte memory blocks. Within this 
its reserved memory space 67 in manipulating data stored in reserved memory space 67, the CPU stores information 
the sequential memory. As an example, if the user selects a representing memory allocation including indices of 
menu choice of deleting a specific form and instance, the memory used by form type, form instance, page number, and 
CPU checks its reserved memory space to locate data in the time of data entry, for processing commands either from the 
data memory 69 corresponding to that data. The CPU is not clipboard (via the user interface) or the remote computer, 
required to physically overwrite data in the data memory, but Apart from the reserved memory space 67, data and 
rather, it changes its reserved memory space to remove commands intended for later use (e.g., page change 
references indicating use of the data memory corresponding commands) are stored sequentially in the remaining avail- 
to the form. Similarly, the CPU processes requests to upload able (data) memory 69 on a first-come, first-served basis. In 
specific form data to the remote computer 61, e.g., by the reserved memory 67, the clipboard CPU also stores 
instructing the sequential memory to only transmit data indications of used and available memory, and displays 
found at selected memory locations in the clipboard. memory status to the user, either upon request or when the 

in. PREFERRED DATA FORMAT memory is fiill. 

If the clipboard 11 is coupled by serial interface to the 

A. STYLUS DATA GENERATION. remote computer 61, the remote computer can command 

At times when the clipboard is awake, the CPU receives 35 uploading of data corresponding to either the entire data 
stylus data as an analog signal, and it processes that data to memory, or to a specific page or form; this command may 
determine stylus position and whether stylus data represents also be effected using the menu functions of the clipboard 
"ink" or other page markings. Beneath the paper supporting user interface 25. Data is preferably transmitted to the 
surface of the clipboard, a crossing pattern of "x" and 'V remote computer in units of "pages," which are delimited by 
driver signals, with signals separated in time and trace lines index entries in the CPU's index list. Within a page, data is 
arranged in a unique configuration permit the CPU to very transferred directly from the clipboard memory to the 
closely detennine stylus position relative to the paper sup- remote computer, via the serial output link, without inter- 
porting surface, even when the stylus is several millimeters pretation occurring at the clipboard. Decompression of the 
away from that surface. For example, the paper supporting data, decoding of the stylus position data, and interpretation 
surface can repeatedly use eight different "x" driver lines 45 of the interpolated context and other input data, are then 
and eight different driver lines in a non-repeating subsequently carried out by the software running on the 
pattern. The writing tip 19 electronically detects these driver remote computer. Once data has been transferred from the 
signals at different times and in varying relative magnitude clipboard, the user is preferably prompted whether to erase 
depending upon distance between the stylus and correspond- data which has been transmitted to the remote computer, or 
ing driver signal. The stylus provides a data signal to an 50 all data which is associated with the particular page or form 
analog-to-digital converter, which the CPU interrogates at instance. If erased, the reserved memory space 67 is used by 
ten miUisecond intervals; during each such interval, the CPU the CPU to locate data and erase data corresponding to the 
determines relative magnitude of each of the sixteen driver particular page or form instance, even if stored in fragments 
signals, and uses a look-up table to uniquely detennine within the clipboard's data memory, 
stylus position. 55 (^3^^ memory is implemented as flash memory, memory 

While the CPU when awake constantly "listens" to stylus reuse is possible if data has been deleted in contiguous 

data (e.g., for reentering a sleep mode foUowing a period of memory blocks (typically 128 K bytes), either at the begin- 

inactivity), the CPU preferably otherwise ignores stylus data ning or end of the clipboard's data memory space. Unless 

unless the tip-down switch within the stylus is depressed, sufficient memory space is available either at the beginning 

indicating that the user is writing upon a surface. At this or end of the clipboard's data memory space, the clipboard's 

point, the CPU performs compression as further described user interface indicates that memory is full, and data must be 

below, and then loads the data into the next memory position uploaded prior to acceptance of additional data. Importantly, 

in sequential memory as a two-to-six byte stylus position a defragmentation feature coiild optionally be implemented 

packet. The configuration of data within the stylus position in clipboard firmware, to compact valid data to fill-in unused 
packet wiU be further described below. 55 space each time data is erased, but this option has not been 

Additional details about the configuration of the preferred implemented in the preferred embodiment. Alternatively, the 

digitizer, and scheme for detecting stylus position, can be user can also add PCMCIA cards, and the CPU is effective 
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to ascertain the capacity of new memory. The occurrence of +/-31 units of the preceding "x" position and is encoded as 

particular patterns in the input data, for example, a signal a signed six-bit delta from the previous "x" position. In this 

from the user interface that a new form has been inserted, instance, the second "x" byte is omitted, as indicated by FIG. 

may cause the CPU to add a pointer into the memory to its 6, and the second data byte is then the first byte of "y" 

index, so that this particular event may be readily retrieved. 5 information. The "x" sign bit is the most significant bit of 

Once the data has been written to the memory, it is prefer- this first 'V* byte, and the magnitude of horizontal position 

ably not subsequently altered or reordered, change is represented in the last five bits of the first data 

Within this framework, the preferred compression scheme byte. If the "delta-x" bit is reset (binary zero), then the "x" 

is explained with reference to HGS. 4^. This compression position is outside the delta range and absolute horizontal 

scheme utilizes data packets having {x,y,t} coordinates and 10 position is encoded as an unsigned 14-bit value using the 

representing either a code or sequential position of the stylus first bit of the third byte and the low 13 bits of the first two 

on the surface of the digitizing pad. In other, alternative bytes. 

embodiments, for example, an embodiment which include a If the "delta-/* bit is set, then the "y" position is within 

built-in microphone, the data packets can also represent +/-63 units of the preceding "y" position and is encoded as 

audio data, with the "t" coordinates indicating data "type" to 15 a signed seven-bit delta firom the previous "y" position. In 

be audio. In the uncompressed state, each packet consists of this case, the second byte is omitted, and the "delta-y" 

six bytes, while in the fully compressed state, a packet can value is encoded in the low seven bits of the second data 

have as few as two bytes. byte, as indicated in FIG. 8. If the "dcUa-y" bit is reset 

In the case of stylus data, the uncompressed data packets (binary zero), then the position is outside the delta range 

include three two-byte words for the "x" coordinate, the "/' ^ and is encoded as an unsigned fifteen-bit value representing 

coordinate and stylus pressure and packet type information absolute position, in the low fifteen bits of the second group 

("t" data). NormaUy, stylus pressure will be indicated via of "y" data bytes (e.g., either the second and third bytes, if 

one or two bits indicating whether the writing tip of the "x" data is compressed, per FIG. 6, or the third and fourth 

stylus is in contact with a surface, and the two bytes bytes otherwise, per FIGS. 4-5). 

otherwise carry identification information as to whether the ^ With this scheme, the large majority of stylus data points 

packet represents either clipboard codes, stylus "type" data, require only two bytes for representation in memory. The 

or other non-stylus data. actual number of bytes used to represent a compressed data 

If the "t" data bytes indicate that the packet represents point can be determined from the three "delta" flags in the 

date or time, the four "x" and "y" bytes are interpreted as the fi^l byte. The scheme can very effectively be used to report 

year, month and day or as minutes and tenths of seconds "x** position across the paper supporting surface with a 

after midnight, respectively. Form identification is per- resolution of one in 16,384 uniU, and "y" position with a 

formed using the four "x" and "y" bytes to represent raw resolution of one in 32,768 units. 

ASCII data, e.g., from a bar code scanner or a form identifier C. TRANSFER OF DATA. 

from the CPU's reserved memory space, and by encoding Data is stored in sequential data blocks, with the CPU 

page number as a separate data packet. using its reserve memory space to store flags identifying 

The {x,y,t} data format is also used to communicate codes forms and page changes and the like; in such a manner, 

not stored as part of the sequential data stream in the either the clipboard or the remote computer (when in com- 

clipboard's memory, generally for communication with the munication with the clipboard) may instruct upload of (a) 

remote computer. The clipboard can encode form index memory contents, including page and form codes, or (b) data 

relative to the clipboard (e.g., one of one-hundred and thirty for specific pages or forms, either using the CPU's index or 

two possible active forms, using the four "x** and "y" bytes a unique code for the page or form (e.g., a bar code), 

to encode time together with the last seven bits of the "t" To upload data using the user interface, the user of the 

bytes carrying form index number). The clipboard can also clipboard places the clipboard in communication with the 

report its memory status to the remote computer using the 45 remote computer which, as mentioned earlier, can be a 

"x" and "Y' data words to indicate the number of utilized personal computer running suitable software. Preferably, the 

(32K) blocks, and the number of bytes in excess of a clipboard includes several alternative mechanisms for 

multiple of 32K, respectively. communication, including a serial connection (e.g., an 

Many other code schemes are available for encoding RS-232 coupling). When turned "on," the clipboard can act 

particular types of non-stylus data, and selection of a suit- 50 in either a master or slave mode for communicating with the 

able coding scheme is left to one famihar with digital remote computer, either permitting the user of the clipboard 

communication schemes. to select either a page index (e.g., page 1-132) or a page 

Preferably, stylus data is compressed to as little as two identification code, including form instance; the user selects 

bytes for minimizing memory use and data transmission appropriate menu options displayed via the LCD of the user 

requirements, using a delta compression scheme. In the 5S interface, and then pushes the "send" button to trigger 

preferred scheme, indicated with reference to FIGS. 4^, the upload to the remote computer. Preferably, an acknowledg- 

highest-order bit of the first byte is the "delta-t" flag, the mcnt from the remote computer is required before upload 

second highest-order bit of the first byte is the "delta-x" flag, can begin and, foUowing upload, data which has been 

and the third highest-order bit of the first byte is the successfully uploaded to the remote computer is automati- 

"dclta-y" flag. Uncompressed data thus includes two bytes 60 cally erased from the clipboard. The user interface and 

each of "x" and "y" position values, and two bytes of packet associated menu preferably also include means for dialing a 

type (or "t") data, as seen in FIG. 4. When the "delta-t" bit phone number and controlling a modem to establish and 

is set (to binary 1), the value of the two "t" bytes differs from maintain communication with the remote computer, 

that of the previous point, hence these two bytes are included The remote computer, once in receipt of an uploaded file, 

in the data packet. If the "delta-t" bit is reset (binary 0), the 65 saves the file as a distinct record on hard disk. This record 

two "t" bytes are omitted from the data packet, as indicated can then be used to (a) print entered stylus data, e.g., using 

by FIG. 5. If the "delta-x" bit is set, the "x" position is within a printer to print directly upon a blank page or blank 
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preprinted form, (b) edit the entered data, for example, using 
the "splicing" utility mentioned below, (c) combine the 
entered data with image data, such that the form can be 
recreated entirely from electronic memory and then printed 
upon blank paper or saved, or (d) suitably archive the 5 
entered stylus data, with an index to the type of form which 
was originally used (e.g., "invoice, 1995 version"); whether 
or not a code identifying page instance specifically identifies 
form type, the remote computer can add such information 
for proper archiving. lO 

IV. "SPLICING" USING THE REMOTE 
COMPUTER 

As indicated above, software on the remote computer 
segregates entered data according to identified pages, such 
that image "windows" are created for each page of data. In 
addition, the software preferably provides utilities for com- 
munications with the clipboard for purposes of checking the 
clipboard's memory status, battery power, index of forms 
(either current active forms or specific "type" and "instance" 
codes), for determining contents of the clipboard's memory, 
and for uploading either all memory or data corresponding 
to a specific form. 

The remote computer also preferably includes "splicing*' ^ 
software which permits a time playback of a segment of 
entered data upon a computer monitor, such that a user can 
advance forward and backward in time and "watch" a time 
window of entered stylus data being visually added to 
displayed images. With the ability to focus on a segment of 
time during which data was entered, the "splicing" feature 
enables a user to separate spatially overlapping data based 
upon relative time of entry, and permits that data to be 
reassigned to different pages of data, or data from different 
pages of data to be combined into a single page. The 35 
"splicing" feature is farther described with reference to 
FIGS. 9-17. 

FIGS. 9-10 each show hypothetical writings entered 
using the clipboard 11 of FIG. 1, where it is presumed that 
a page change was not electronically marked in the aggre- 40 
gate data stream. Illustrated via FIGS. 9-10, the user has first 
entered a monetary amoimt ($69.95) 101 upon a top page 14 
(FIG. 9), then switched to a bottom page 20 (FIG, 10) to 
enter the text "Have a nice day" 103, and lastly, switched 
back to the top page (FIG. 9) to enter two initials ("tj") 105. 45 
In this example, a single data stream was created having 
stylus data representing (in order) (a) ten millisecond sam- 
plings of the drawing of the monetary amount (including 
time and date codes), (b) ten millisecond samplings of pen 
strokes corresponding to the text "Have a nice day" 50 
(including time and date codes) and (c) ten millisecond 
samplings of entry of two initials (including time and date 
codes). When processed by software on the remote 
computer, a resulting image 107 has the appearance indi- 
cated in FIG. 11, and it is desired to sort entered data into 55 
two image pages (as the data was physically entered on 
different pages). 

In this regard, if proper electronic page markings had been 
entered at a significant location within the sequential data, 
the sequential data would have included the following (as 60 
well as appropriate time and date codes): (a) a command 
identifying the top page; (b) ten millisecond samplings of 
the drawing of the money amount ($69.95); (c) a conmaand 
identifying that the user now wishes to write upon the 
second page; (d) ten millisecond samplings of pen strokes 65 
corresponding to the text "Have a nice day," (e) a command 
identifying that the user now wishes to write upon the top 
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page; and (f) ten millisecond samplings of entry of two 
initials ("tj''). The "spHcing" feature of the software is relied 
upon to properly sort the data, in the absence of properly 
entered page identification codes. 

FIGS. 12-18 are used to explain the operation of the 
"splicing" feature used by a remote computer in the previous 
example, to separate the spatially overlapping data and form 
two separate pages from that data. The operator of the 
remote computer first uploads the entered stylus data from 
the clipboard (reported as corresponding to a single form) 
and selects an appropriate time range and the "splicing" 
feature. 

As seen in FIG. 12, the "splicing" feature causes the 
software to display a graphics window 109 for recreating 
and displaying the act of entering data upon the multiple 
forms, from an electronic viewpoint. A status bar 111 at the 
top of the FIG. 12 indicates that time is "early" in the writing 
process represented by FIGS. 9 and 10, e.g., corresponding 
to a time prior to any writings on either the top or bottom 
pages. FIG. 12 also indicates the presence of "fast rewind," 
"play backward," "play forward/' "fast forward," "pause," 
"mark" and "cut" buttons 113, 115, 117, 119, 121, 123 and 
125, which are displayed at the bottom of the screen. An 
operator of the editing software utihzes a mouse or other 
interface to control the buttons to move time forward or 
backward from with respect to data entry upon the clipboard, 
much in the same manner as playing a video tape. If time is 
moved backward, images already displayed upon the screen 
will be erased in the reverse order in which those images 
were entered, sampled-point by sampled-point. If time is 
played forward, additional images will be displayed in the 
order and lime in which they were entered upon the clip- 
board. 

FIG. 13 is a representation similar to FIG. 12, but in which 
the "play button" 117 has been pressed and some time has 
passed, as indicated by the status bar 111 at the top of FIG. 
13; the sequence of data entered using the clipboard has been 
"played forward" to correspond to the time just after entry 
of the written amount 101 ($69.95, seen also in FIG. 9). It 
is at this point that the user of the clipboard switched to 
writing upon the bottom page, but since no page identifica- 
tion appears in the data stream, the editing program as 
additional time passes will erroneously superimpose addi- 
tional data (e.g., the text "Have a nice day") upon the 
amount. Thus, FIG. 14 represents the remote computer's 
graphics display window 109 where additional time has 
passed, corresponding to the writing of both of the amount 
($69.95) 101 upon the top form and the text "Have a nice 
day" 103 upon the bottom form. 

The operator of the remote computer then selects the 
"pause" button 121 to stop the addition of further writings 
which are later in time, and also selects the "mark" button 
123, which is used to electronically mark a point in time with 
respect to the writings. As seen in FIG. 15, the operator then 
chooses the "play backward" button 115 of the graphics 
display window 109, to thereby move backward in time 
while marking for edit the erroneous data, e.g., the super- 
imposed text "Have a nice day." Since each datum from the 
input device (e.g., the stylus) is associated with a specific 
time, and each sample from the input device is kept in order 
in a sequence, overlapping visual data may be distinguished 
notwithstanding any spatial overlap of that data. Once the 
operator of the computer has moved sufiScienUy backward in 
time, as seen in FIG. 16, the operator stops the playback by 
selecting the "pause" button 121. At this point in time, the 
just-removed text "Have a nice day" has been marked based 
on time parameters entered by the operator, and only the 
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monetary amount ($69.95) remains displayed in the graphics associated relative times, notwithstanding any spatial 

display window 109. [Notably, the graphics display window overlap in spatial coordinates between the subset of 

depicts the top form and the amount ($69.95) in their proper data and the remaining data when displayed, 

relative position and orientation, as designated by the ref- 2. An apparams according to claim 1, wherein the data is 
erence numeral 110.] 5 stylus stroke data and includes data representing at least two 

The operator then selecU the "cut" button 125 (FIG. 17), complete stylus strokes, each stylus stroke being associated 

which preferably causes the opening of a new, "cut" window with a time of entry, said instructions further oontrolUng the 

127 for a separate image (i,e., the cut writings "Have a nice machine to: 

day"). The operator can then create a second graphics select, separate and display such that at least one complete 

display window 129, corresponding to a separate hardcopy stylus stroke is selected and separated from the data, 

page, into which the cut writings are copied (FIG. 18). This and displaying the remaining data without the at least 

new window can either contain an existing image along with one complete stylus stroke which has been selected and 

the copied image superimposed with the correct spatial separated. 

orientation, or a new electronic display page created by the 3. An apparatus according to claim 2, wherein the stylus 

operator; also, as with the first graphics display window 109, stroke data includes samples taken at periodic intervals, the 

the second window 129 displays the cut data in its proper samples representing continuous stylus motion, each sample 

spatial (x,y) location and orientation with respect to a form, associated with a time of entry, and wherein said instructions 

exactly as it was physically entered on the clipboard, e.g., as control the machine to: 

represented by a form designated 130 in FIG. 18. accept inputs from the operator that represent relative 

Finally, the operator then again selects the original "play" time parameters for contiguous stylus data within one 

button 117, to move time forward and display additional stroke; and 

writings in the first graphics display window 109 as they select, separate and display such that the contiguous 

were originally entered (FIG. 18). The operator at this point stylus data is removed from the first graphics display 

is left with two separate images, one corresponding to entry window. 

of data upon the top page (FIG. 9) and one corresponding to 4. An apparatus according to claim 1, wherein the relative 
entry of data upon the bottom page (FIG. 10). Notably, in the 25 Umes represent time of associated data entry, said instruc- 

example just illustrated, writings were marked and cut in tions further controlling the machine to: 

reverse time sequence, based upon parameters entered by the select relative time parameters via 

operator; the operator could have marked and cut writings in displaying data according to time of entry, and 

forward time sequence as well. accepting inputs from the operator and responsively 

. ^. , J. . f.i. *• *• 30 causing the display of data such that data is played 

Vanous alternative embodiments of the present invention - 5 -^u * * c , u *u * #u 

, . 1 -11 • 1 . • forward with respect to time of entry, such that the 

will occur 0 those havmg sbll in electromcs or software. ^^^^ simulated to the operator at approxi- 

For example, another input device for creating spaUaUy ^^^^^^ ^^^^ ^^^^^ ^^^^ originally entered, 

variant data may be used instead of a stylus and digitizing 5 ^ apparatus according to claim 1, said instructions 

clipboard, such as a mouse or other unplement. Software f^^^^j. controlling the machine to: 

alternatives will also occur to those having programming provide operator-selective placement of the subset of data 

skiU, which effect splicing of input-data based upon time, to ^ second graphics display window, such that the 

sort that data between different image windows or other gy^set of data is displayed in the second graphics 

applications; all of these modifications are viewed as within display window, but not the remaining data, 

the spirit of the present invention. 6. An apparatus according to claim 5, wherein said 

Having thus described several exemplary implementa- instructions further include controlling the machine to: 

tions of the invention, it will be apparent that various provide the operator with abihty to copy a second subset 

alterations, modifications, and improvements wiU readily j^ta from the first graphics display window to the 

occur to those skilled in the art. Such alterations, second graphics display window without removing the 

modifications, and improvements, though not expressly 55^^^^! ^^^^^ ^j^ta from the first graphics display 

described above, are nonetheless intended and implied to be window. 

within the spirit and scope of the invention. Accordingly, the 7 ^ apparatus according to claim 1, wherein the instmc- 

foregoing discussion is intended to be illustrative only; the ^^^^^ further control the machine to display operator selec- 

invention is limited and defined only by the following claims ^^j, buttons of play, mark and cut, and to accept operator 

and equivalents thereto. selections of said operator selection buttons. 

What is claimed is: 8. An apparatus according to claim 1, wherein the instruc- 
1. An apparatus comprising instructions stored on tions further control the machine to display operator selec- 
machine readable media, wherein the instructions cause the ^qjj buttons of play forward, play backward and pause, and 
machine to permit an operator to edit data based upon select operator selections of said operator selection but- 
relative times associated with that data, the instructions jqjjs 

controlling the machine to: 9. A method of editing data representing spatial 

associate the data with a first graphics display window; coordinates, wherein the data is associated with relative 

accept inputs from the operator defining relative time times, the method using a computer processing device, an 

parameters; operator interface, a display monitor, and at least a first 

select a subset of the data based upon inputs from the graphics display window, said method comprising: 

operator; associating the data with the first graphics display win- 
separate the subset of data from the data, to leave remain- dow; 

ing data; and selecting relative time parameters corresponding to the 

display the remaining data (without the subset of data) in data via the operator interface; 

the first graphics display window; 65 separating a subset of data from the data, the subset 

wherein the subset of data in the first graphics window differentiated from remaining data based upon the 

may be removed from the remaining data based upon relative time parameters; and 
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displaying the remaining data (without the subset of data) 
in the first graphics display window; 

wherein the subset of data may be removed from the 
remaining data (of the first graphics display window) 
based upon associated relative times, notwithstanding 
any spatial overlap in spatial coordinates between the 
subset of data and the remaining data when displayed 
together. 

10. A method according to claim 9, wherein the data is 
stylus stroke data and includes data representing at least one 
complete stylus stroke, the at least one stylus stroke being 
associated with a time of entry, said method further com- 
prising: 

selecting, separating and displaying such that at least one 
complete stylus stroke is selected and separated from 
the data, and displaying the remaining data without the 
at least one complete stylus stroke which has been 
selected and separated. 

11. A method according to claim 10, wherein the stylus 
stroke data includes samples taken at periodic intervals, the 
samples representing continuous stylus motion, each sample 
associated with a time of entry, and wherein; 

selecting relative time parameters includes selecting con- 
tiguous stylus data within one stroke. 

12. A method according to claim 9, wherein the relative 
times represent time of associated data entry, and wherein: 

selecting relative time parameters includes displaying 
data according to time of entry and providing an 
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Operator with ability to play the data forward with 
respect to time of entry, such that the display monitor 
and computer processing device display to the operator 
simulated entry of data at approximately the rate at 
which data was originally entered. 

13. A method according to claim 9, further employing a 
second graphics display window, wherein said method fur- 
ther comprises: 

placing the subset of data into the second graphics display 
window; and 

displaying in the second graphics display window the 
subset of data, but not the remaining data. 

14. A method according to claim 13, wherein said method 
further includes: 

providing an operator with ability to copy a second subset 
of data from the first graphics display window to the 
second graphics display window without removing the 
second subset of data from the first graphics display 
window. 

15. A method according to claim 9, wherein the selecting, 
separating and placing are performed under the control of an 
operator via controls including play, mark and cut. 

16. A method according to claim 9, wherein the display of 
data upon at least one of the graphics display windows is 
performed under the control of an operator via controls 
including play forward, play backward and pause. 
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